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The work presented in this paper investigates the use of metaheuristic 
optimization algorithms for the face recognition problem. In the first setup, a 
face recognition system is implemented using particle swarm optimization 
(PSO) and firefly optimization algorithms, separately. PSO and firefly are 
used for forming the feature vectors in the feature selection stage. These 


feature vectors serve as the new representation for the face images that will be 
fed to the classifier. In the second setup, selected features from both PSO and 
firefly algorithms are fused to form one single feature vector for each face 
image before the classification stage. Extensive simulations are conducted 
using Poznan University of Technology (PUT) and face recognition 
technology (FERET) face databases. Optimal values for population size and 
maximum iterations number were selected before conducting the experiments. 
The effect of using different numbers of selected features on the performance 
is investigated for feature selection using PSO, firefly, and feature fusion of 
both. 
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1. INTRODUCTION 

We live in a world where analyzing an enormous amount of diversified data is becoming a crucial and 
fundamental necessity. Therefore, in the analysis, we can bring out proper conclusions from the data we possess 
and classify it into positive, negative, or neutral sets. It is evident that people around us affect our daily life. 
People tend to change their minds and decisions based on other people’s opinions and decisions. To some 
extent, we use other people’s ideas to come up with our own decisions. Based on this idea swarm intelligence 
concept was introduced [1]-[3]. It is a nature-inspired artificial intelligence based on the communication 
models of social insects such as ants [4], [5], fireflies [6], [7], and bees [8], [9]. 

A swarm is a large number of agents interacting locally with themselves [10]-[13]. In a swarm, there 
is no supervisor or central control to give orders on how to behave. Swarm-based algorithms are popular in this 
era with the thirst for nature-inspired, population-based algorithms that can generate low-cost, fast, and correct 
solutions to complex and hard-to-solve problems. For that reason, swarm intelligence is becoming a golden 
ticket in the era of artificial intelligence metaheuristic optimization [14]-[16]. Swarm intelligence optimization 
techniques have been applied successfully in numerous applications where they helped to improve the system 
performance such as proportional integral (PI) controller tuning in wind turbines [17], the maximum power 
point tracking (MPPT) enhancement in photovoltaic systems [18], and cervical cancer classification [19]. One 
of the applications that the optimization algorithms can further improve its performance is face recognition. 
Face recognition is a very hot research topic especially with the increasing demand for security in different 
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areas such as businesses, public transportations, borders, and airports. Since the start of using computers to 
recognize human faces around sixty years ago [20], many algorithms and techniques have been introduced to 
improve face recognition performance in challenging scenarios such as different poses, lighting, orientations, 
and facial expressions [21]—[24]. Hermosilla et al. [25] proposed to fuse thermal and visible descriptors where 
the system obtained the optimal weights using the particle swarm optimization (PSO) algorithm to maximize 
the face recognition rates obtained from different combinations of local descriptor methods. PSO was applied 
to coefficients extracted by the discrete cosine transforms (DCT) and the discrete wavelet transform (DWT) 
algorithms where it searched the feature space for the optimal feature subset based on a discrimination 
criterion [26]. In this paper, we propose to use two well-known optimization algorithms namely; PSO and 
firefly optimization algorithms for improving face recognition accuracy. In this paper, these two algorithms 
will be applied at the feature selection stage of the face recognition module in two different setups. In the first 
setup, the optimization algorithms are applied to face images for features selection and their performances are 
evaluated separately, whereas in the second setup the features selected from both optimization algorithms are 
fused before evaluation. The performance of the proposed approach will be tested using two popular face 
databases. 

The rest of the paper is organized as follows: section 2 presents the optimization algorithms used for 
feature selection. Section 3 describes the methodology and setup. Simulation results are shown and discussed 
in section 4. Finally, the paper is concluded in the last section. 


2. RESEARCH METHOD 
2.1. Optimization algorithms 

In this work, two swarm intelligence metaheuristic optimization algorithms are employed for feature 
selection. The selected salient features are used for class description and discrimination. The working 
mechanism and detailed explanations of both particle swarm optimization [3], [27]-[30] and firefly 
optimization [6], [7] algorithms are presented briefly below. For more details, reader is encouraged to refer to 
respective references. 


2.1.1. Particle swarm optimization (PSO) 

This algorithm has been introduced by Kennedy and Eberhart [3]. It can be summarized as: given that 
the swarm has a certain number of particles, every single particle in a population has a current location, a 
current speed, and a personal best location in a search area. The personal best location is related to the location 
in the search area where an objective function provides the minimum calculated error for the corresponding 
particle. The location corresponds to minimum error among all the personal best locations is declared to be the 
global best location. Both, personal and global best locations and speeds are updated for every particle in the 
population at each iteration using the equation in [3]. 

An inertia weight, adapted by PSO, is linearly decreased during training to control the convergence 
rate of the algorithm. The next position of the particle is determined by accumulating the new speed to the 
particle’s current location. Limiting the speed vector value to a predefined range will guarantee the particle 
does not leave the search area. A detailed description of the algorithm can be found at [3]-[5], [31]. 


2.1.2. Firefly optimization 

The firefly optimization algorithm is a global metaheuristic optimization algorithm, proposed by 
Yang [6], which imitates the behavior of firefly insects. Fireflies use the flashing behavior to attract other 
fireflies, usually for sending signals to the opposite sex. However, in the mathematical model, used inside the 
firefly algorithm, simply the fireflies are unisex, and any firefly can attract other fireflies. 

Firefly brightness amount is the key factor for its attraction by other fireflies. For a couple of fireflies; 
the brighter one will attract the other; so, the less bright one will move in the direction of the brighter one. This 
is done on every iteration of the algorithm for any binary combination of fireflies in the population. For more 
details about the algorithm, readers should refer to [6], [7], [32], [33]. 


2.2. Methodology and setup 

At the first setup, we implemented our system using PSO and firefly optimization algorithms 
separately. PSO and firefly helped in the feature selection stage of the system. These selected features were 
used as a new representation for the images. They are fed to the classifier to evaluate the performance of the 
system as shown in Figure 1. At the second setup, after separately applying PSO and firefly to the image, 
selected features from both algorithms were fused to form one feature vector, as shown in Figure 2. The rest 
of the system is similar to the first setup. 
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Two face databases were used to evaluate the proposed approaches. The first database is Poznan 
University of Technology (PUT) database [34]. It contains images of 100 people. Each person has 100 images 
covering varied facial expressions and orientations of size 128x128 pixels. 10 images per person with a total 
of 1,000 images are used in our experiments. The second database is face recognition technology (FERET) 
database [35]. A subset of FERET database with 194 people is used in our experiments. This subset contains 
10 images per person of size 96x64 pixels. Examples of faces from both databases are shown in Figure 3. 
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Figure 1. Block diagram of the proposed PSO/firefly-based texture classification 
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Figure 2. Block diagram of the proposed feature fusion face recognition system 


Figure 3. Face examples from PUT database (on the left) and FERET database (on the right) 


The optimal swarm size (population) is problem-dependent which describes the social interaction 
within the swarm. While smaller populations are slower in convergence, they are less likely to fail into local 
minima and have more reliable convergence to optimal solutions [3]. Starting the search with small populations 
and increase population size proportionally to increase in iterations number ensures an initial high diversity 
with faster convergence as particles move towards a probable search area [36]. 
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The number of iterations to reach a good solution is also problem-dependent. Too few iterations may 
terminate the search prematurely. On the other hand, a too large number of iterations adds avoidable 
computational complexity (valid if the iterations number is the only stopping criteria) [3]. In this study, 
different population sizes and number of iterations were evaluated before deciding their optimal values for the 
rest of the simulations. 


3. RESULTS AND DISCUSSION 

As mentioned earlier, two face databases are used for performance evaluation. We investigated 
different population sizes and number of iterations to find the optimal values for the rest of the simulations. 
Optimal values for all simulations are chosen as 20 for population and 100 for the maximum number of 
iterations. k-nearest neighbors (k-NN) is used with three different distance metrics namely; Manhattan, 
Euclidean, and Cosine. 10-fold cross-validation was applied in all the experiments. Results recorded in all 
figures and tables are averaged ones. Tables 1 and 2 show performances by applying different numbers of 
selected features for PSO and firefly algorithms, using PUT and FERET databases, respectively. 

Firefly algorithm is superior to PSO algorithm in all scenarios in both tables with different numbers 
of training images and selected features. According to the results in Tables 1 and 2, Manhattan distance gave 
the best performance. Based on this, results in Figures 4 and 5 are reported only using this distance. 


Table 1. Recognition rates with different number of selected features of PSO algorithm using PUT/FERET 
face databases 


g 2 # Training Images 
3B DB 
g 4 1 2 3 4 5 6 7 8 9 
lı 45.56/20.25  55.84/26.84 64.96/33.57 71.95/41.28 75.00/38.33 77.30/39.51 — 80.73/46.55 79.40/41.52  79.80/45.57 
10 l 40.78/16.13  52.17/21.91 60.76/28.83  67.52/36.05 71.12/32.28 74.22/33.94  77.67/39.98 —76.20/36.78  76.80/39.64 


Cos 33.31/12.61 _43.69/17.32  52.83/21.88  59.08/29.29  64.00/25.66  66.45/26.53_ 68.40/32.49 _68.45/28.84 —_ 68.50/32.63 
lı 59.46/36.55 74.88/45.23 82.17/55.41 87.62/64.24 87.90/65.35 90.45/72.28 92.67/71.98 94.55/72.01 94.80/74.69 
20 dl 51.74/27.29 66.72/34.28 75.53/44.79 80.72/51.22 81.92/53.48 86.13/58.38 88.90/58.80 91.20/58.80 91.30/61.08 
Cos 45.43/23.52 _60.61/30.01 _70.19/39.96 _75.88/46.39  76.94/48.31 81.22/53.79 _ 84.33/53.80  87.30/52.63  88.00/57.47 
lı 70.10/51.32 83.16/62.74 90.30/75.71 93.10/79.75  94.78/86.88 95.67/88.44 97.37/87.94 97.65/89.87 — 97.90/91.24 
50 dl 62.72/40.35 76.81/50.39 84.54/64.27 88.47/69.45 90.78/76.40 92.15/78.98 95.00/78.33 95.90/80.93  96.60/82.58 
Cos _55.48/34.64 _71.47/44.76 _79.91/58.84  85.03/64.47 _87.36/72.00  89.67/74.30  93.13/73.77 _ __93.75/717.16 _94.70/78.35 
lı 74.86/64.42 86.34/74.46 91.67/84.22 94.85/88.38 96.36/90.39 97.17/93.65 98.20/94.40 98.30/94.51 98.20/94.69 
100 i 68.02/50.93 80.60/62.07 87.67/73.73 92.18/80.11 94.20/83.77 94.78/87.50 97.17/88.83 96.85/88.22 97.30/87.94 
Cos 61.90/44.97 _75.20/56.92  83.91/69.42  89.27/76.14 92.06/80.03  93.30/84.11 95.23/85.00  95.40/84.90  96.90/85.26 
lı 79.13/71.00 90.36/79.79 93.71/88.82 95.97/92.27 97.00/94.09 98.08/95.77 98.90/96.49 98.35/96.78  98.90/97.27 
250 l 73.40/56.95 86.80/67.35 91.16/79.06 93.87/85.38 95.38/88.09 96.95/91.60 98.00/92.13 97.90/92.37  98.30/93.61 
Cos 67.59/51.40 83.14/62.30 88.49/75.41 91.52/82.44 93.62/85.67 95.40/89.30 97.13/89.52 97.20/90.26 97.60/91.91 
lı 82.51/72.40 92.19/81.60 95.51/89.65 97.20/93.63 97.70/95.25 98.38/96.30 99.23/96.92 98.85/97.35 99.10/97.99 
450 dl 76.88/58.12 88.59/69.32 92.79/80.50 95.52/86.86 96.60/90.06 97.10/92.82 98.57/93.51 98.25/93.45 98.40/94.38 
Cos 71.64/52.71 _ 86.10/64.88  90.76/77.13 94.10/84.20  95.36/87.68  96.03/90.82  97.77/91.32 _97.85/91.49  98.40/92.94 


Table 2. Recognition rates with different number of selected features of firefly algorithm using PUT/FERET 
face databases 


g 2 # Training Images 
3B RZ 
3 4 1 2 3 4 5 6 7 8 9 
lı 49.30/23.24 63.44/29.36 73.07/37.94 76.50/43.69 80.36/45.03 80.20/45.57 —86.87/47.04 84.95/51.55 87.90/51.75 
10 l 43.10/18.54 57.64/23.80 67.97/30.60 71.35/36.03 75.08/40.18 76.17/39.48 — 82.23/40.79  80.75/44.69 — 84.80/43.45 


Cos 37.09/15.83  51.39/20.14 _60.41/26.50 _64.22/30.51 _69.12/38.06  68.75/36.29  75.23/33.56 _73.60/38.87 _77.90/37.32 
lı 62.71/34.94 77.05/43.05  85.17/52.17  88.88/60.84 89.84/66.45 92.20/70.67 94.77/68.78 —95.10/69.97 —96.00/69.12 
20 dl 54.17/26.08  68.60/33.80 77.96/42.06 82.08/49.33 84.18/53.99 87.90/58.30 —90.50/56.79 91.60/58.81 — 91.70/57.99 
Cos 48.54/22.12 _62.41/29.50 _71.96/36.69 _77.67/43.99 _79.52/49.09  83.83/51.87 _ 86.90/51.94  88.30/53.12 _ 88.10/51.75 
lı 73.98/52.69 — 85.53/63.36 91.56/75.53  94.08/82.77 95.46/85.42 97.10/89.79 — 97.97/87.37 97.90/89.30 —98.10/90.72 
50 dl 64.48/40.18 78.71/50.05 85.97/62.50 90.00/71.45 91.92/75.40 93.63/80.05 94.97/76.70 95.25/79.05 — 97.20/81.08 
Cos 58.66/35.69 _ 73.55/44.99  81.87/57.26  86.25/66.44  88.84/70.95  91.20/76.16 —_93.23/72.32 _94.05/74.95 _95.10/77.84 
lı 78.34/63.12 89.97/75.21 93.74/83.59 95.40/88.89 96.98/92.05 97.88/93.07 98.83/94.24 98.50/93.92 98.90/94.64 
100 dł 70.94/48.91 85.16/62.47 90.26/72.64 92.53/80.25 94.82/84.54 95.72/87.04 97.33/88.33 97.40/87.27 98.20/87.78 
Cos 64.81/43.39  81.16/57.02 87.19/68.45  90.08/76.49  93.12/80.81  94.15/83.72 _96.50/85.15 _96.20/84.05 _97.50/84.95 
lı 81.94/69.24 92.36/79.00 94.81/87.95 97.13/92.66 97.48/94.02 98.08/96.16 99.03/96.43 99.10/96.42 99.10/96.96 
250 l2 76.34/55.14 88.42/66.57 92.10/78.34 95.25/85.37 95.78/88.10 96.75/91.47 98.10/92.49 97.90/92.76 98.10/93.20 
Cos 70.62/49.81  85.33/61.44  90.01/74.65  93.67/82.18  94.70/85.54 _95.65/88.85  97.27/89.48 _97.25/90.64 _97.50/91.19 
lı 82.34/71.68 92.21/81.37 95.37/89.79 97.32/93.49 97.96/94.98 98.35/96.47 99.20/97.16 99.20/97.37 99.10/97.89 
450 l2 77.28/58.05 88.81/69.76 92.67/80.66 95.37/86.77 96.46/89.55 97.40/92.86 98.53/93.26 98.45/93.81 98.70/94.85 
Cos 712.22/52.35 86.41/64.95 90.87/77.22 94.08/84.04 _95.02/87.38 96.45/90.57  97.77/90.91  97.85/92.27  98.00/92.84 
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Figure 4 shows a comparison among PSO, firefly, and PSO+firefly algorithms using PUT database. 
Fifty percent of images per person were used as a training set and the rest were used as a testing set 
(500 training images/500 testing images). A similar comparison was conducted using FERET database with 
970 training images (Fifty percent of images per person) against 970 testing images. Figure 5 shows the results 
of this comparison among PSO, firefly, and PSO+firefly algorithms. 
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Figure 4. Performance comparison with different number of selected features among PSO, firefly, and 
PSO+firefly (feature fusion) algorithms using PUT face database (500 training images/500 testing images) 


In both figures, it is clear that using more selected features helped to improve the performance using 
either PSO or firefly. For the fusion of PSO-generated features and firefly-generated features, improvement 
was obvious with a fewer number of selected features. For example, Figure 4 shows that with 20 selected 
features using PSO and firefly algorithms, performances reached 87.9% and 89.84% respectively. The 
performance increased to 94.44% with feature fusion/concatenation (20+20=40 fused features). The increasing 
number of fused features above 200 fused features (100 from PSO and 100 from firefly) did not show much 
performance improvement as shown in Figure 4. 
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Figure 5. Performance comparison with different number of selected features among PSO, firefly, and 
PSO+firefly (feature fusion) algorithms using FERET face database (970 training images/970 testing images) 


Figure 5 shows similar results using FERET database even though the whole performance was lower 
due to the use of 194 people with 1940 images in total. Using 20 selected features from PSO and firefly algorithms, 
performances reached 65.35% and 66.45%, respectively. The performance increased to 82.25% with feature 
fusion/concatenation (20 + 20 = 40 fused features). Similar to the results of PUT database, it is clear that increasing 
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the number of fused features from both PSO and firefly beyond 100 features each, will not have a noticeable 
effect on the performance. It will only increase computational complexity. 


4. CONCLUSION 

We investigated the use of two of the metaheuristic population-based optimization algorithms namely; 
PSO and firefly, for the face recognition problem. PSO and firefly optimization algorithms are utilized in the 
feature selection stage for generating the feature vectors. The application of such algorithms showed 
enhancement in the recognition performance of the system. Firefly algorithm provided better performance than 
PSO algorithm. In addition, the fusion of selected features from both algorithms forming one single feature 
vector further improved the recognition performance. Results indicated that using optimization algorithms for 
feature selection is a good choice for improving the performance with much fewer features. Furthermore, 
feature fusion of the selected features generated using both optimization algorithms helped to boost the 
performance with fewer selected features. 
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